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ABSTRACT 

A study vas conducted by the Air Force to determine 
the extent to vhich takeoff/landing skills learned in a simulator 
eqtiipped vith a night visual system voiild transfer to daytime 
performance in the aircraft* A transfer-of ^training design tras used 
to assess the differential effectiveness of simulator training vith a 
day versus a night coiiputer*ge!ierated image <CGI) visual display* 
Twenty^four novice student pilots were divided into three groups: 
day, night, and control* The day and night groups received three 
trainiu: missions in the Advanced simulator for Pilot Trailing (ASPTl 
cn takeoff, straight--in approach and land/ and touch*and*go* The 
ccntrcl groT3p received standard syllabus instruction <that is, no 
ASPT)* Transfer to the aircraft was assisted by instructor pilots on 
tvc aircraft sorties performed during daylight conditions* Results of 
the study indica'.^d that (1» student performance improved 
significantly on the takeoff, full stop straight*in, and the takeoff 
portion of the touch*and--go{ (2) there vere no differences betveen 
the day and night groups as assessed by instructor pilot performance 
ratings^ and (3) there vas no tendency for performance to be 
differentially influenced by the differences in day and night scenes* 
(Author/LIS) 
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COMPUTER GENERATED IMAGE; RELATIVE 
TRAINING EFFECTIVENESS OF DAY 
VERSUS NIGHT VISUAL SCENES 



L INIRODUCHON 

The Air Force v& in theproccssof increasing iu&itnulator training capabilities for most weapons 
systems to inclnde contactAriSnal flight training. In some cases* entirely new simulators are being 
designed; in other cases, existing devices are bemg improved. In either case, an important area of 
consideration is the choii e of a visnal display system. Numerous options for simulator visual systems 
are commercially available. Major system types include TV Model Boards Compnter^rencrated 
Image (CGI)^ and Point Light Source systems. Across and within each system type^ a variety of 
choices can be made with respect to acqnisition costs and the ability to meet training objectives. 
Although information is available to specify system costs^ little information is available to determine 
training benefits. 

The research was undertaken to determine the extent to which takeoff/landing skills learned in a 
simultor eqnipped with a night visnal system would transfer to daytime performance in the aircraft. 
The possibility that learning to fly with the aid of a night vienal scene simulator would result in flying 
habits and strategies specific to nighttime flying raises questions regarding the extent and direction of 
training transfer that can be expected for daytime flight training. An area of particular concern is in 
the critical stages of flight where any negative transfer might be particularly hazardous, especially for 
the novice pilot. However^ if snbstantial positive transfer could be achieved nstng a night scene (at 
least comparable to other types of displays)^ considerable cost savings conld be realized. Training 
effectiveness data are needed in order to supply information reqnired in a cost-to4>eneftt analysis 
model, particniarly when the choice is between an add-on or retrofit modification to an existing 
device versus procnrement of an entirely new device. 

Background 

Additional htizards arc associated with night flying and have been well docnmented in the 
aviatiot literature. In addition to vertigo and spatial disorientation, poor final approaches have a 
higher probability of occurrence at night. In a snrvey conducted for the Navy, Simons (1965) 
reported a day^ight carrier landing accident ratio of 1:4. In snbsequent research on dayAiight 
carrier approaches^ Brictson (1966) reported that pilot tended to fly lower approaches and to land 
longer and soher at night than by day. The most revealii^ measures were those of variability, with 
the standard deviations of the night approaches more than twice that of the daylight approaches. 
This situation resnlted in the necessity of attempting to make large corrections in the Tinal stages of 
the approach^ an obviously undesirable predicament. 

Although the problems with night landings may be more critical for the carrier situation, they 
are of no less concern to the general aviation community. The specific behavioral problems may 
differ between carrier lauding$t and runway landings (Lewis & Humphries^ 1956)^ but one trend is 
consistent, pilots fly lower with much greater variance in their night approaches. The tendency to 
lo-id short on runwayj as opposed to long on carrier decks has been reported in the safety litcratnre. 
It is not clear whether this is act-ially representative of typical night approaches or whether it is only 
representative of th€ unsuccessful ones. 



The ptlofs inability to make accurate judgements of altitude, sink rate^ and rate of closure was 
identified as a major contributing factor in many pilot-error landing accidents (Zeller* 1957). At 
night, the difficulty of making these judgements is increased due to the absence or degraded nature of 
the environmental cues. This situation has led to the development of many runway and cockpit 
landing aids. However, the last portion of the approach is dependent on visual flight restructions and 
pilot judgement. 

Advances in modertv visuat display systems have made it possible to simulate a variety of 
environmental conditions including ntght scenes. It is now possible to practice night flying in the 
safety of a simulator. It is also possible (a) to study problems involved in night flying, (b) to 
determine the visual cues required for effective Emulation, and (c) to develop effective training 
techniques aimed at increasing transfer benefits and safety. 

Research efforts have begun in an attempt to improve visual scene content and to define the 
training value of various visual display systems. Kraft, Anderson, and Jlworth (1977) has 
demonstrated that pilots tend to overestimate their altitude when lights in a computer-generated 
night scene have equal luminance, regardless of distance. The flightpaths under these conditions 
were significantly lower than when the luminance of the lights was attenuated. Theoverestimation of 
altitude was greatest when no texturing cues were added to the overrun and runway scenes. K raft et 
al. (1977) a!so reported that runway position at touchdown improved with practice with the 
attenuated luminances hut that the luminance and texturing variables did not result in improved rate 
of descent at touchdown 

Bucklaifd (1979) has investigated the landing performance of expert- enced pilots in the 
Advanced Simulator for Pilot Training (aspt) under various detail and texturing conditions. Two 
of the conditions included night scenes similar to the one used in the present study. The results 
indicate that night scene touchdowns were harder than any of the day scene conditions. The night 
scene touchdowns were also farther from the thrcehold than were the average day scene touchdowns. 
Buckland also reported that the touchdown vertical velocity values were substantially larger for all 
of the scene conditions than for actual flight data. This finding is consistent with the results reported 
hy Palmer and Cronn (1976) in a study of touchdown performance in a DC-8 simulator equipped 
with a night computer graphics visual attachment. The authors also reported that the vertical 
velocity values obtained in thetr study were approximately 10% to 20% lower than values obtained 
in an earlier expcri- mem using a TV model system (Bray, 1972). 

The primary intent of the previously cited research was to study the characteristics of night (roal 
or simulated) flying and the problems associated with night flying. More directly relevant for the 
purposes ui the present study is research dealing with transfer of training from night scene 
simulation to daytime aircraft flight. To date,* only two studies have addressed this issue;; (a) the 
Navy (Brictson & Burger, 1976) conducted an evaluation of the A-7E Night Carrier Landing 
Trainer (NCLT) which is equipped with a night visual scene, and (b) the Air Force (Thorpe, 
Varncy* McFadden^ LcMasten & Shorti 1978) conducted a comparative evaluation of three visual 
system types forpossihlc application tn the KC*135 program. In the transfers f-t raining evaluation 
of the Navy A-7E NCLT conducted by Brictson and Burger ( 1976), half of the subjects received 80 
NCLT approaches prior to actual flight training. Their performance was compared with a control 
group of pilots who did not receive NCLT approach training. Transfer evaluations were made hoth 
during the field carrier landing training and carrier qualification phases on both night and day 
approaches. Measures of performance included radar readings landing signal officer scores, sucttess/ 
attrition rate, wire arrestments holtcr rate« and landing performance scores. The results indicated 
that NCLT training resulted in significantly better performance on night approaches^ particularly on 
vertical path control, had more impact for new pilots than for pilots with other aircraft bacic* 
grounds* and resulted in significantly lower attrition rates (8% vs 44% )> However, the NCLT 



trainmgdid not improve performance on daytime appro aches, and this is the most important finding 
for the present study since it shows the transfer of training was specific to the nighttime environment 
and did not generalise to the daytime condition. 

Thorpe et aL (1978) conducted a transfer of training experiment comparing three vi&ual 
systems: a TV Model Board, a Daylight Color CGI« and a Night-Only Point Light Source CGI. Three 
groups of pilots recently graduated from the Undergraduate Pilot Training (UPT) program received 
simulator training on the visual traffic pattern* approach, and landing using one of the three visual 
systems (using Boeing 707 aircraft simulators rented from commercial flight training sources). 
Following the simulator training, the pilots flew two evaluation flights in the KC*135 aircraft in 
daylight conditions. Although there were no significant differences observed during the simuhor 
training* the results of the evaluation flights indicated that the pilots in the Day Color CGI Group 
and the Night-Only CGI Group performed better on the final approach gUdepath and landing 
portions of the task than did the pilots trained with the TV Model Board system. The latter group 
exhibited more extreme deviations from the glidepath* especially lower glidepaths iluring the final 
third of the approach than did the other two groups. A follow-up comparison of their scores in the 
CCTS program revealed that a higher percentage of pilots in the Day Color CGI Group received 
^'Highly Qualified*' (90%) scores than did those in the Night (50% ) or TV Groups UM ). Only 
30% of normal CCTS students received this grade. 

ThuSf while there are simulator performance data and subjective opinions indicating that CGI 
systems (both day and night) lack adequate cues for altitude, sink rate, and closure rate estimation, 
the only transfer-of-t raining research data available indicate that the CGI systems (color day and 
point light night) are at least capable of supporting daytime trainings and may be superior to the TV 
model board. The Navy study demonstrated positive transfer from night simulator training to night 
carrier approaches^ but no effect was observed for transfer to day approaches. Equally important for 
the purposes of this report is the finding that no negative transfer was evident in the performance of 
the night group on subsequent day performance in either study. 

Ihe Pkobkm 

The Air Training Command has recently introduced a new ground*ba$ed flight trainer, the 
Instrument Flight Simulator (IFS), into the UPT program. Although its primary purpose is to 
support instrument skill acquisition^ a secondary purpose is to support limited contact training 
objectives. Currently* half of the IPS cockpits are equipped with a look-ahead field-of-view TV 
terrain model board visual system. Prior to making any further procurements for the remaining 
cockpits* the Air Training Command requested an estimate of the amount of transfer of training 
which could be achieved with a dusk/night limited field-of*view visual system. The use of such a 
visual system could result in significant cost savings if iherc was no loss In training effectiveness. Of 
particular interest was the issue of negative transfer potential in the critical phases of flight, i.e.* the 
takeoff and landing area. The present research was undertaken to answer this question, as well as to 
provide a comparison between the relative effectiveness of the ASPT CGI day fccne uitli the night 
scene. 

H MEIIIOD 

Geneml Approneh 

A transfcr-of^raining design was used to assess the <liffcrential effectiveness of simulator 
training with a day vs. a nighl CGI visual display. Twenty-four novice student pilots were divided 
into three groups: Day, Night, and Control. The Day and Night groups reccivcti three training 
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missions in the ASPT on Takcoffn, Straight^In Approach and Landing* and Touch^nd4?o. The 
Control group received standard syltabus instruction (i.e., no ASPT). Transfer to the aircraft vas 
assessed hy Instructor Pilots (IPs) on two airciafi iiorties performed during daylight conditions. 

Subjects 

Twemy-four student pilots participated in this study. Twelve students each were selected from 
UPT Class 7842 and 7843. The following descriptive information characterizes the samplc^ (a) 
Source of commission — 11 Reserve Officer Training Commission, 13 Air Force Academy 
graduates; (b) no previous navigator experience; (c) X age ^22.8 (s.d. 1. 12, range 22—27); (d) 
previous simulator experience, T =0.2S hours (s.d. 0.84> range 0—3) and (e) previous aircraft 
experience^ X ^2.7 hours (s.d> 17.8^ range 16.2 — 40). None of the students had previous jet aircraft 
experience. 

bistnic tor Pilots 

Four experienced T-37 IPs served as simulator instructors during the prctraining phase of the 
study. The flight instructors normally assigned to the students served as data collectors during the 
aircraft portion of the study; none of the ASPT instructor pilots participated during this phase. 

AH participating flightline instructors received familiarization training iu the ASPT on data 
collection procedures to be used in the aircraft. This training coasisted of a verbal briefing followed 
liy practice in data taking in the ASPT. Each IP viewed two performances of a Takeoff* Full Stop V 
Straight-In Approach and Landing^and a Touch-and^ro. One performance was representative of an 
experienced T-37 IP and the other of a novice pilot. These performances ' ^cre prerecorded and 
stored for demonstrations; thuis, each pilot viewed the same st^t of maneuvers. The instructors were 
thoroughly debriefed^ and definite errors were clarified. The IPs were also given a written 
information sheet to serve as a refresher prior to inflight data collection (see Appendix B). The 
entire training period was approximately L5 hours in length. The training was given 10 days prior to 
the actual aircraft data missions. 

Eqtiipmtnt 

The ASPT research equipment provides a wide range of capabilities not previously provided in 
any one simulator A complete description of ASPT is presented in Cum« Albery* and Basinger 
(1975). An overview of the aspects of the ASPT most relevant to the present study is presented in 
this section* 

The ASPT is equipped with two T-37 cockpits. Each cockpit has a full fietd*of-view visual 
display (300** horizontal by 150** vertical), a CGI visual system, a six-degrecs-of-rreedom (DOF) 
synergistic platform motion system^ and a 16-panel pneutnatic C-scat on the left seat (student 
position). 

The visual display is projected through seven cathode ray tubes (CRTs). The capacity for 
displaying visual image detail is fixed and shared between the two cockpits. A highly detailed scene, 

, such as an airport^ requires 90% to 100% of the display capacity; thus^ at the most^only 10% ofthe 

capacity would be available to the other cockpit. This amount would result tn inadequate 

i representation of a highly detailed scene but is adequate to display a generalized view from altitude 

such as a horizon and surface texture pattern. The ^^isuat system uses an infinity optics display with 
the exit pupit located at the student's eye position,. This arrangement results in an optimal visual 
scene from the student position^ but a distorted scene from the IP position. From a normal position, 
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the IP Ls unable to sec the visual display immediately m front of the aireraft. The scene becomes less 
distorted as the IP scans laterally. By moving the head position nearer to that of the student's, the IP 
can increase the forward looking view and reduce the distortion. 

The platform motion system is driven by six hydraulic actuators, each with a travel capability of 
60 inches. The platform motion system software was designed to provide translational and rotational 
acceleration onset cues to the student pilot position. The C-seat can also display sustained 
accelerating cues: however, the C-seat was not used in this study and will not be discussed. The 
motion system also includes a special effects package which is used to display such cues as touchdown 
bump, runway rumble, aircraft buffet, speedbrake extension, and gear-down rumble. 

The .ASPT has the capability of real-time automated measuremetit of the pilot's performance. 
Measurements can be made of pilot inputs, system outputs, and scores can be derived from these 
measures. The measurement schemes or algorithms for a given maneuver can be preprogrammed. 
An entire sortie content can be preprogrammed whh automated performance measurement taking 
place on predesignated trials. A limited amount of this information can be displayed real-time in the 
cockpit via a monitor located to the right of the IP position and/or following the mUsion in hard 
copy form. 

The ASPT is equipped with the capability of displaying a prerecorded demonstration of a 
laneuver. The information is stored on disc and replay involves reproduction of the entire 
aneuver as originally recorded, including visual display, motion cues^ instrument readings, rudder 

and throttle movements. Both the students and the IPs utilis^ed this capability during the pretraining 

and ASPT phases of this study. 

Wsiial Display 

The data base developed for the night visual study consistf of a night airport traffLC urea 
configured to closely approximate the .McDonnell-Douglas Vital 111 display and equivalent day 
airport traffic area. 

A compntcr-controlled mask restricted the field of view to ^8^ horizontally aitd 36*^ vertically. 
\ generaliz^'d airport scene was crcatrd wjth one model depicting the night scene and another model 
depicting the day scene. For the night scene, the environment tiicludetl a 6,000-foot lighted runviay 
with approach lighting nitd titnway markings. The ruiiwa) centertine was visible for UOOO to 2,000 
feet in front of the aircraft on the riinwav, with touchdown zone stripes at the approach end of tlic 
nmway. Inset runwa) lights were also provided to simulate the rnnway ligliting of the McDonnell- 
Douglas Vital III systciu. There were l>jsc lights random l> positioned out to 10,000 Tccton both sides 
of the runway and from 10.000 feet out from the departure end of the runway. Horuou lights were 
also randomly positioned at a further distaneci providing the appearauie of convergence as the 
aircraft descended to lower \\s\ii\\ approach angles. Horizon glow was added providing a dusk/niglit 
scene. Other visual d^'tails included a cube tower with rotating beacon on top. offset 5.000 feet to the 
left uf the runway. There were two radio towers uffset 3,000 feet to the right of the runway and two 
light towers un the right side of the ruuwaj offset 2.000 and 1.000 feel from the end of the ninwa). 

The day scene consisted of a full) marked (>.000 foot ruuwu) with rnuway edge Hghti^ and 
sequenced flashing approaching the runw<i>, Low level \elocit) and altitude cue enhaneement wa^ 
achlc\cd through tlie use of numeroui^ three^iimen^onal featnrvs in this aa^a. On final iipproaeh. 
there wa^ a drIve-in movie theatre and an agricultural arta consisting of a t hiirth, farm huildtng.s, 
pk-knp truck, wagon^and tractor with plow/fhe control tuwer iind other building!) wen' adj:nent to 
the runway, with a faetor) in the bai kgroond, dud at the thresholds a To7 aircraft W(ti> uaitiug tu take 
the active runway^ 




Student Haining 

Following sdection for study participation^ the subjects were randomly assigned to one of three 
groups: Day, Night, or Control. Subjects in the Control group did not receive any ASPT pretraining 
prior to entry into the T-37 flying phase. Students in the two experimental groups received three 
ASPT sorties in ^hich instruction was given on Takcoffs, Full Stop Straight-In Approaches, aud 
Straight-la Approaches to a Touch-aad-Co Takeoff. 

The visual field of view was limited to 48^ horizontal by 36** vertical by a con) putcr-gcaerated 
mask. This was essentially a look-ahead view designed to represent the field^f-vicw available on the 
IFS in use by the Air Training Command. The ASPT syllabus consisted of three sorties, eaeh 
approximately one hour in length. The first and second sorties were separated by a4&*hour interval, 
with the second and third sorties occurring in daily succession. The content of each sortie was 
specified in terms of the number of repetitions per task and task order. The ASPT syllabus is 
presented in Appendix A. A total of 13 Takcoffs, 13 StraiglitJn Approach/Landings, and 9 Straight* 
In Approach/rouch*and«Co maneuvers were performed. Following completion of the three ASPT 
uiissionsand standard Syllabus requirements, the students began flightlinc training. The desired time 
between the last ASPT missioti and the first aircraft flight was 1 day; however, in some eases the 
interval was as loug as 3 days because ASPT training was coniplctcd on a Friday. 

Ihinsfer Evaluations 

The first T-37 aircraft flight was conducted as spceiHcd in the standard UPT syllabus with the 
proviso that a demonstration of a Takeoff^ Straigbt-In Approaelu and Full Stop Landing and a 
Straight-In Api>roaeh to a Touch*an<l«Co Takeoff was given by the IP with no "'hands on" practice 
by the student. 

The second and fifth T-37 flights served as thedataeolleetiou missions for all three groups. The 
IPs were asked to design eaeh mission to include at least ouc hands^n repetition by the student of 
the Initial Takeoff, Straiglit-ln Approach to a Toueh-and'Co. and a Straight-In Approach to a Full 
Stop. Performance evaluation data wrre to he recorded on each task by the IP as soon as possible 
following task eompletion. In the event that more than one repetition of any of the tasks was 
performed, the IP wasaskcd to eolleet the desired data on these additioual repetitions as well. The fP 
was asked to refrain from instrueting during the performanee of eaeh task: however^ in no ease was 
the safety cf the fliglit to be eompromised. In the event that the IP had to take control of the aircraft, 
only the portions of the flight flown by the student were to be reeonled. 

Dependent Measures 

Performance measures of two types wi^rc collected on each trial designated as a measurement 
trial. (No performanee measures were collcctcti on the reriiainmg trials.) Root*mcan-s<|uarc error 
measures were eolleetetl on eaeh task parameter which had a eriterion-refcrcnccd objective 

In addition to the objective antoniated meat^urcii, stibjcetive performance ratings were obtained 
from the IP. The basic rating scale w^is the .same used in the normal ATC training program^ This 
sealc_Spccifics standards for each grade of V (Unsatisfactory). V (^*air)♦G (Good) ^ and E (Exeellenf). 
For the pnrposcs of ihh study, this seate was subdivided into an Absohite scale* corresponding to 
published perfonuance standards, and a Uelative scale. The IP was tnstnictcd to use the Kclattvc 
scale by assigning a grade basc<) upon a comparison of the t^tudent's performance to that of other 
studtmts at the same point in train in{^. The standard of comparison for the Relative sealc was totally 
subjeetive^ based on the IP's aeenmulated tcarliing experience. This dual standard scale was nsed in 
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an attempt to reduce tlie variance typically encountered in rating scale judgements. Previous 
researcli suggested tlie use of t lie dual scale would provide Increased measurement sensitivity as well 
as reducing inter- and intra-rater variability. 

Tlie original study protocol included use of tlie task observation data to be collected In botli tlie 
ASPT and tlie T-37. However, during IP pretraining sessions^ it wassliown to be extremely difficult 
to make accurate observations of runway alignment from the riglit (IP position) of tlie ASPT due to 
tlie distortion of tlic visual display from tliat position. Tliercforc.tliesc Jata were collected only on 
tlie two T-37 evaluation fliglits. The data card formats are presented in Appendix C. 

Data Analysts 

Tlic automated data collected in tlie ASPT was analyzed using a multivariate analysis of 
variance teclinicfiie (MANOV A) wliicli provided an overall test of significance (WilkuHLambda) and 
univariate stepdown F tests for eacli of tlie individual parameters. Tlie RMS error scores were 
logarithm ieally transformed prior to analysis. Due to several bad data points and tlie unavailability 
of appropriate statistical programs, eacN measured repetition of eacli task was subjected to a separate 
MANOVA. The confidence limit established for tlie Wilks-Lambda was p <.10. 

Relative and Absolute scale ratings of task proficiency given in tlie U, F, G, and E format were 
transformed into integer values 1* 2, 3, and 4, respectively. For ratings collected in tlie ASPT, data 
from eacli scale type were analyzed separately using a mixed design analysis of variance (ANOVA)^ 
From tlie ratings collected in tlie T4?, only Relative scale data were subjected to statistical analysis 
since visual inspection of tlic Absolute scale data rrwcaled no mean differences witli almost no 
variability. Due to incompiete data, a simple ANOVA was performed for cacli repetition of eacli task. 
Tlie confidence limit was set at .05. 

Tlieoriginalintent was to use aebi square analyds for cacli item of tlic categorical data collected 
on tlie evaluation fliglits. However^ tliore were too many cells containing expected frequencies of 
zero. Therefore, tlie entry for eacli item from eacli task on cacli fliglit was converted to eitlier 
"correct" or "incorrect." Tbus, cither a "liigli" or *1ow'' data entry was treated as **in correct,^* Tlic 
frequencies of "correct*^ and "incorrect"* items were summed across aU tlic items forming tlic basis of 
a two (Correct vs. Incorrect) by tlircc (Day vs. Niglit.^vS. Control) contingency table. A separate 
c<]»ntingency table and associated analysis was computed for caeli task on each fliglit. Tlic cbi square 
statistic tests the likeliliood tliat tlie observed distributions were due to cliancc. Tlie coefficient of 
contingency, Ct was also computed for eacli table. Tliis measure gives an indication of tlie degree of 
association between tlic groups and tlielr performance^ 



OL RES y LIS 

ASPT Data 

Three questions are of interest when examining tlie outcpmcs of tlie ASPT train! - (a) Did tlie 
students* ptjrformancc improve during the ASPT training? (b) Were there an) prrformunce 
differences as a function of ttic day versus night scenes used during training? U) Did the students* 
performance improve differentially as a function of the day versus nif^lit scenes? Two types of 
dependent measures are available to address these questions; IP performanrc ratings and automated 
system state measures of task perform n nee. 
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Tabic 1 presents mean values of the performanee ratings by tasL group, seale type^ and 
repetition number. These data were analyzed using a mixed design analysis of variance (ANOVA) 
with Day versus Night groups as the between-group factor ^nd trials as the repeated measures factor. 
The resulting F statistics and associated probability levels are presented in Table 2. 

Table 1 - Instructor PUots* Mean Proficiency Ratings : ASPTlVBining 













Tb»12 




TrittS 


Ta»k€ondiiiait 




Relative 


Absolute 


Relative 


Absolute 


Relative 


Absolute 




Day 




1.25 


2.875 


2.125 


3.125 


2.50 


Takeoff 


















Night 


2.75 


1.50 


2.875 


2.125 


3.25 


2.75 


Full Stop 


Day 


1.873 


1.375 


3.375 


2.50 






Straight-ln 














Approach 


Night 


2.50 


1.875 


3.00 


2.50 






Approach 


Day 


2.875 


2.50 


3.50 


2.875 






Portion of 
















Touch-and-Go 


Night 


3.125 


2.50 


3.125 


2.625 






Takeoff 


Day 


2.75 


2.125 


3.25 


3.0 






Portion of 
















Touch-and-Go 


Night 


3.125 


2.25 


.3.125 


2.75 






Tabic 2. ASPT Training; H* Bnlings - F Statistics 


Task 




Cn>up (P vs 


N) 


Triib 




Cioup xTriibt 


Takeoff 




B 


<i 




4.550* 




<1 






A 


<i 




18.034* 




<1 


Full Stop Str-!u 




R 


<i 




11,200* 




2.80 






A 


<i 




13.451* 




1.098 


Strain App 




R 


<i 




I MO 




1.000 




A 


<i 




<1 




<1 


Toueh^and-Go 




R 


<i 




2.333 




2.333 






A 


<i 




11.930* 




<1 
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The data collected by the ASPT aulomated performance nLcasurcmejit (APM) system ^ere 
analysed using a MAN OVA. The means and standard deviations of ihe APM data and the results ot 
the associated staiistieal analyses arc presented by task in Tables 3 (TakeofOt^ (Full Stop Straighi-ln 
Approach) t and 5 (Straight-!n Appruach/Touch^and-Co). Due to occasional equipment failure^ some 
of the daia were lost. For this reason^ each repetition of each task vas analysed separately. Rather 
than estimating mission data, the <lata were deleted for a subject selected at random from the 
uppusite group. In these casos* n =7 instead of 8. 



Table 3. Descriptive and Inferential Statistics 
AS PT/Auto mated MeasuresrTikeoff 









IleadliiiF 


Attinitle 


Altitude 






Speed 


Envr 


Ermr 


Enor 


IHal 1. 












Day X 


77.62 


87.74 


.968 


.860 


536 


SD 


6.S3 


2.38 


.756 


.292 


.56 


Nighi X 


75.14 


84.74 


..574 


.878 


5.59 


SD 


6.71 


4.16 


.828 


.504 


.39 


^'(1.12) 


<1 


2.728 


<1 


<1 


<1 


n =7, Lambdii 












TriaI2 _ ' 












Day X 


78.06 


86.65 


.660 


.842 


5.38 


SD 


6.29 


5.36 


.521 


.471 


73 


Nighl X 


77.29 


86.91 


.296 


.620 


5.17 


SD 


9.23 


6.92 


.504 


.266 


.49 


14) 


<1 


<1 


2.016 


1.350 


<l 


n ~8. Lanibila 


-WilksF-. <l 










IHal 3 












Day X 


78.44 


88.03 


.354 


.577 


5.U 


SD 


T..12 


4.35 


.508 


.335 


„3;j 


Nighl X 


72.26 


84.50 


.411 


.278 


5.20 


SD 


5.36 


4.31 


.480 


i.24 


.57 




3.711* 


2.670 


<1 


<l 


<1 


n =8, Luinltdii 


-\VilksF(5 ,0) <l 











Table 4, Descripttve and bferential Statislks ASP 'Autonuted Measuros 
PuD Stop Stm^hirbt AppiouvA 



FUml Appioteh 



Aliviude Cenicriioe 
Etior Em>t 



Day X 4.30 4.56 

Sp .45 .44 

Nighi X 4.23 4.73 

SD .38 .38 

P{I,12) <I <I 
n =7, Lambda -Wilks F(gj) =1.203 p <.402 
1Hal2 

Day X 3.86 4.39 

sp .36 .57 

Night X 3.42 4.60 

SD .63 .46 



P(l,14) - 2.860 <1 

n =8. Lambda - Wilks Fj^^^j =2.569 p <.098* 
'p <.10. 



GUdepatb 



Air Speed GlSdeitdi Cenleriuie Air Speed 
Emr Enor „ Enor Ennr 



1.18 


.274 


3.32 


1.56 


.76 


.692 


.94 


.82 


.88 


-.429 


3.92. 


I.I3 


.77 


.526 


.55 


.68 


<I 


4.584* 


2.088 


1.150 


1.19 


-.539 


3.08 


.737 


.37 


.624 


.62 


.577 


.92 


.018 


3.51 


.732 


.55 


.424 


1.04 


.326 


1.363 


. 4.373* 


1.07 


<1 
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Table 5* Descriptive and bifetendal Statistics 
ASPT/Autamated Measui^s Stiaight-In ^pioach/Touch-and^rO 



FimtApp Ccnteilitie Final App GUdcpftth C«nteiliii« GUdfpftih Hfftdbg Atiiiud« TftkcotiT 
All Em>r £mr A/S Em>r Emr hnar A/S Enor Emr Emr All Emr 

(LOGRHS) aX>G RMS) (LOG RMS) OX) G RMS) O-OG RMS) (LOG RMS) (M>G RMS) (M)G RHS) (M>G RMS) 



Tiial 1 





3,7S 


■t,3^ 


JB 


*.308 


3,S9 


J23 


,71 


,82 


5,49 


Sl> 




,S2 


,78 


,635 


,45 


,36 


,59 


38 


,69 




3,79 


4,8:! 


,53 


-,092 


3,89 


,99 


,35 


,94 


5,49 


Sl> 






,41 


,42 


,64 


,63 


,38 


,21 


.40 


*'(IJ4) 


<l 


.1,232 


<l 


<1 


1,083 


1,099 


:!,LI5 


<l 


<1 






<l 
















rmi:2 




















Day X 


3.-11 


4,53 




-.41 


3,34 


,ao 


,56 


j(i9 


541 


SD 


,47 


,33 


,3S 


,44 


,6J 


.56 


,67 


,19 


,55 


Ni(il»i ? 


3,(*3 


4,S(> 


,68 


*^3 


3.35 


j69 


,71 


,87 


5,15 


SI) 


,S7 


,3a 


^6 


,69 


,87 


^9 


47 


,22 


38 


^(ia« 


<l 


<l 


\M2 


<l 


<l 


<l 


<l 


3.093 


1,291 



<I0, 
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The results of the performance rating analyses provide information with reapect to all three of 
the questions of intiirest, whereas the analyses of the APM data are only relevant to the Day versus 
Night comparisons. The results of the ASPT training jihase may be gummarixed as follows, (a) 
Student performanee improved significftrtly on the Takeoff^ Full Stop Straight-In, and the Takeoff 
portion of the Touch*and«Co. (b) There were no differences between the Day and Night groups as 
assessed by instructor pilot performance rating. Analyses of the APM data indicated initial 
superiority of the Night group on the single parameter of control on thcglidepath angle of descent on 
the Full<Stop Straight-In Approach. However^ the Day group was significantly better than the Night 
group on that parameter and the task as a whole by the next measurement point. There were no 
reliable performanee differciices on the Takeoff or Toueh^and'Co. (c) There was no tendency for 
performanee to be differentially infl\iencc(l by the differences in day and night scenes. 

T-37 Tinansfer Evaluation 

Tlicre are two cjuesUons of primary interest eonccrning student performance in the aireraft. (a) 
Were there any reliable diffcrcnees between the performance of the Day and Night group? (b) Did 
either or botli of the ASPT trained groups perform better than the Control group. Questions 
coneeniing negative transfer indications (or the Night group and learning differences between the 
three groups are abo of iiitcrcst. Deviations in the study protocol oecurred whieh affeeted the ability 
to address these questions as originally planned. These items will be diseussed briefly before 
proeeeding with a presentation of the data. 

As originally designed^ the transfer evalua.ionS\vere to eonsist of at least one repetition of eaeh 
tri^fk on eaeh of the two evaluation flights. However, for various reasons, these goals were not 
eonsistently aceompHsheil. The biggest problem oeeurred on the Approach portion of the Full Stop 
orToueK-and-Oo tasks. Apparently^ tho students were given the opportunity to fly One or the other^ 
but not both maneuvers. The IPs were mojre willing to allow the students to fly the takeoff portion of 
the Toiieh-and«Co. As a eonsequenee, the data from the Approach segments of the Full Stop and 
Toueh*an(l«Co were eoUapsed and treated as the same task. Another eonsequenee of the ineomplete 
data eoUeetion was the inability to use a mi.xed design A NOVA on the rating data. Separate ANOVAs 
of each repetition were used instead. 

Two types of information were eolleeted by IPs: (a) relative and Absolute performanee ratings 
as used in the ASPT {ihase^ and (b) criterion -referenced observations of aireraft control. (In some 
rtistaneeSt the IP marked more than one alternative for a given item > e.g., circled -50i 0, and 50 feet 
on altitude control. When this oeeiirrcdi eaeh eircled option was treated as an integer for eaeh 
category in the 'Contingency tables.) Tables 6. 7, and 8 present the 2 by 3 eontingency tables and 
associated statistics. Talile 9 presents the descriptive and inferential statisties for performance 
ratings. 

Day vs. i\'ighL The resuhs of tlic a priori *'t" tests comparing the {>erformatiee ratings 
(Uelative scale only) <ire the only direct sotiree of information regarding the diffcrcnees between the 
Day and NigEit groups. The results of these analysi^s indieate that there were no signifiennt 
differences U'lv een the groups on any of the tajiks on either evaluation flight. 

Train 'ng Effectiveness. Com|mrison of the performance of the Control group which did not 
■ rceeivc ASPT training with the performnnce of the ASPT -trained students provides the indieation 
^f training effectiveness. The results of the a priori tests of the Day and Night groups combined 
versus the Control group, the *"F" tests, and to an extent, the chi square analyses provide information 
regarding thin issue. However* the rating data obtained on the Absolute scale is equally valuable" in 




Table 6. Task Item Analysis: Takeoff 







Correct 


biGorrect 


a* 


PifSt observation 








Control 


10 


38 




Night 


16 


31 




Day 


15 


32 


df 


=2,x^ =2.33 {p <.50),C =.13 








second observation 








Control 


14 


34 




Night 


27 


21 




Day 


28 


22 


df 


=^2,x^ =8.75 {p <02),C =,24 








i able 7. Task Uern Analysis 


* 






Stniffht-ln ADDioach 










C<»rreci 


[ncomct 




rifSt nnJimfSttAn ' 

M, U 9 1 VMtiS\ri W R t evil 


*■ * 






Control 


17 


55 




Night 


22 


52 




Day 


28 


44 


df 


=2,x^ =4.00 (p <.10),C =.13 






b. 


Second observation 








Control 


27 


51 




Night 


35 


43 




Day 


26 


33 


df 


=2,x^ =8.75 (p <02),C =.10 
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Table S.Tisklieni Analysis: Toueh*and-Go/ 
IV keoff Segment 



a. First observation 
Control 

Night 
Day 

dl ^2, =2.29 (p <.50), C 

b. Seconal observation 
Control 

Night 
Day 

dl =2, =10.96 {p <.01)*C =.2^t 



Table 9. bistnietor Pilot Me«n Rutings and Analysis 
T-37 



Task 




Scale 


C^nlio] 


N^ht 




F 


t(p v» N) 


. T(pyN vs C) 




I 


]t 


1.625 (8)* 


2.125 (8) 


2.7H (7) 


3.638** 


1.279 


•2,2099** 


Takeoff 




A ' 


1.0 


1.0 


1.25 


NC** 


NC 


NC 




2 


H 


2.00 {q} 


2.625 («) 


2.857 (7) 


2:041 


<I 


•1.986** 






A 


J. 50 


2.12 


1.75 


NC 


NC 


NC 




1 


It 


1.625 (8) 


i.7.'>a («) 


2.125 (b) 


<l 


<I 


<l 






A 


1.0 






NC 


NC 


NC 


Approach 


2 


n 


2.125 <8) 


2.750 W 


2.429 (7) 


<1 


<I 


J.l.'>7** 






A 


1.37 


2.00 


1.57 


NC 


NC 


NC 


laktroff 


1 


H 


1.6 (5) 


1.857 (7) 


2.20 to) 


<1 


<l 


<1 






A 


J.O 


1.0 


1.0 


NC 


NC 


NC 


of Touch- 


2 


]{ 


1.87S (8) 


h857 (7) 


2.167 (6) 


<I 


<l 


<! 


and-Ou 




A 


1.120 


10 


1.33 


NC 


NC 


NC 



*$siniplc sixe, 
'**i\[it roinjmliMl. 



8 
14 
14 



26 
21 



14 
26 
11 



42 
31 
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this regard. As far a& the Ahsotutc rating is concerned, there was virtually no positive (or negative) 
transfer of training effeei. The results of the "t" and **F'* tests on the Relative rating data reveal Jiat 
positive transfer oeeurred onU on the Takeoff task. From the ehi square analyses, the only instance 
whieh revealed a non-rhanee distribution was the Takeoff portion of the Toueh-and-Go task. 
Although more difficult to intti^rpret, tt would appear that the performance of the Night group was 
superior {q that of the Control gn)up. This difference is not consistent with the ordinal rankings of 
the groups on the rating data. 

With respect to negative transfer, there were no indications from the data or from tlie postflight 
interviews with the IPs that any potentially hazardous flying skills resulted from the ASPT training 
(Day or Night conditions). 

Learning. Due to the incomplete data returns, the statistical techniques used do not provide a 
direct test of the magnitude of skill acquisition between the two evaluation flights. It is evident from 
a visual inspection of the data that all three groups improved. The more interesting experimental 
question concerns the differential learning rates that may be associated with the ASPT train mg,;>6r 
5e,or the type of visual display, h appears that the Day and Control groups improved approximately 
equivalent amounts from the first to second evaluation flight* with the Night group showing slightly 
larger gains for the Takeoff and Straiglit-ln Approach tasks on the Absolute scale. 




iV. DISCUSSION 

Thcstnily was designed to provide preliminary information regarding the transfer of training of 
a restricted ficld-of-view night visual scene for application to the IPS used for UPT. In addition, a 
comparison was made between the night scene and the day scene available on the ASPT. Both visual 
scenes were produced by CCl system. 

In order to determine the transfer value of tho night scene or to assess the differential 
effectiveness of the day versus night scenes* transfer of training must first be demonstrated. This was 
accomplislied by comparing the performance in the aircraft of students who were trainc<l in the 
ASPT (Day or Night condition) with the performance of students who did not receive ASPT 
training, i.r.^ the Control group. The firS' issue^ then, to be addressed concerns the evidence that any 
transfer of training actually occurred. 

The data on the perform ance ratings provided on the Absolute scale d<i not sho\%' any difference 
in perfoniianre of the three groups. The initial performance of all three groups on all three tasks was 
jndged to be nnsatisfactor). Although performance ratings increased snnicwhat by the fifth flight, 
there was v<^ry liitl<* dirfcrcnee bel\%'crn the three groups. Thus, according to this data $onrec« onv 
would liavC to eoncliidc that no traDsfer of training occiirrcd. Tliedal;) obtained on the It dative scale 
and the task observation information reflect greater sensilivil) to group differences and training 
effects. The results of analyses on these data indicate that the ivro grotips trained in the ASI'T 
performed the 1 akeoff task reliabl) better iban did tbc Control group. Their perfornianee utt the 
other tasks tendrd to be rated higher than the Control group but these differences were not 
Slntislieally sigiiifieant. The niosi reasonable eoncliiiiion regarding the eMeiit of iraiisfer 
demonstrated in this *4i]<ly \%'onl<l be that a small positive transfer t^'ffeet ^as demonstrated for the 
Takeoff task. 

The relatively small >nagnilude of demonstrated transfer is partt<ularl) surpribing in light of the 
faet that significant improvement in pc^rforniam e ^as dcmoiiblratrd b> both e\perun{*nt;]l gnMips 
dnring the ASI'T training phase. By th<' end ofilie ASPT training, slitdeiU perfornianee %a!, rated 
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high in the Fair range on the Absolute scale and in the Good range on thf; Relative scale. Thus, while 
student performanee and skill acquisition seemed satisfactory during the ASPT trainings the 
performance levels were not sustained in transition to aircraft flight. Clearly, the transfer was not on 
a 0ne40<^ne basis as many training program designs expect prior to the introduction of a modern 
simulation device into a training program. 

If transfer is less than the oncfor-Onc tradeoff or if the simulator training does not elevate 
performance in the aircraft by at least a full grade on the operational grading scale, the device is often 
considered worthless by tlie operational training community. However, often times such 
disapgiointing findings arc the result of inadequate performance assessment techniques and of 
underestimating the importance of training technology. The operational grading scales are generally 
designed to detect large changes in profLciency and are, by design, relatively insensitive to smaller 
changes in performance. Thus, while simulator training may not result in a full grade change in 
performance (as from Unsatisfactory to Fair), the training may elevate performance within a 
category {from the bottom to the top of a grade category). These differences may have significant 
pragmatic implications for a training program, especially if similar increments occurred 
continuously throughout cacli block or phase of a s)'llabus. However, by many current operational 
measurements standards, these effects would not be detected. This kind of problem is particularly 
critical to <Jesign and procurement decisions based on the results of tlic Operational Test and 
Evaluation process. Tlie results of the present study reflect a small transfer effect which in itself may 
be trivial but could have a significant cumulative effect when spread over the entire basic contact 
j>hase of training. The remits of this study also demonstrate the difference in sensitivity of varying 
assessment techniciucs. 

With respect to the effectiveness of the night scene for application to daytime flight training, the 
results show that tlie Night group consistently performed better than the Control group although the 
difference's were i;ma)l for the Straight-In anil Touch-and-Co. There were no indications of 
hazardous elements of either the glidcpath or flare segments of the approach. However, given the 
relatively low level of transfer obtained in the present study, it is difficult to draw Rrm conclusions 
regarding the training potential of the night scene. Improvements in sim ulator training strategies ean 
reasonably be expected to enchace the magnitude of transfer* but it is not clear to what extent. It is 
difficult, if not impossible, to make an evaluation of any training device independent of the training 
mctliodology used in the evaluation, 

\ comparison between the night scene and the model board scene would be particularly useful 
considering the resultsof tlie Navy NCLT study (Brictson 8l Burger, 1976) and tliatof the Air Force 
KC-135 dudy (Thorpe ct al., 1978). The iNavy study demonstrated a large pontive transfer effeet 
from the simulator to night flight conditions bnt no transfer effect to da) time conditions I.e., transfer 
s|>eeine to night flight. The Air Force study offered evidence that the model board system was 
inferior to day and niglit CGI systems on the final approach maneuver Since the Air Force study did 
not eni]>loy a control group, it is not possible to assess tlie overall magnitude of transfer. However, 
comparisons were matle late in the operational training program with a pseudo-control indicating 
that all the groups which reieived the simulator pretraining performed better than pilots wbo had 
not particij>ated in the study. .Additionally, it was found that the day-color CGI grouji had 
inaintainctl a superior level of performance wben compared to the night antl model board groups^ 

Considering the data available to date, it is unclear how effective a simulaicd night visual $ccne 
is for transfer to day flight. The present study demonstrates a sniall level of transfer; the KC-^135 
show^ a larger level of transfer: the Navy A-7 study did not demonstrate any positive daytime effect, 
bnt tlierr was a signifLoant nighUinic effect, Ec^ually as ]>crtincnt for Air Training Command 
eonoems i*; the fart that tlie model iHiard sysitcni, the type currently in use on the iKS. was not as 
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effective as the night seene in the KC-135 study. Therefore, themost logieal step for Air Training 
Command would be to eonduet a direet mode] board/night seene eomparison vrithin the operational 
training eoatext. 

V. CONCLUSIONS/RECOMMENDAHONS 

1. A small positive ;ransferK>f-t raining effeet was observed for both the Day and Night 
experimental groups. 

2. There were no reliable differences between tlie Day and Night groups in terms of their 
pcrformanee in the aircraft^ 

3. There was no evidence of negative transfer associated with the performance of the night 
group. 

4. In order to maximize transfcrK>f-training benefits, implementation of a night visual scene 
should be associated with more extensive simulator training than was provided in the present study. 
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APPENDIX Ai ASPT SCENARIO CONTENT 



M£SION 1 



TAKEOFF 
TAKEOFF 
TAKEOFF 
STRAIGHT-IN FS 
STRAIGHT-IN FS 
STR'AIGHT-IN FS 
TAKEOFF 
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DEMONSTRATION 
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APM 
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APM 



MISSION 2 



TAKEOFF 2 

STRAIGHT-IN FS 2 

STRAIGHT-IN TG DEMONSTRATION 

STRAIGHT-IN TG 3 

STRAIGHT-IN TG APM 

STRAIGHT-IN FS 2 



MISSION 3 



TAKEOFF 3 

TAKEOFF APM 

STRAIGHT-IN FS 4 

STRAIGHT-IN FS APM 

STRAIGHT-IN TG 4 

STRAIGHT-IN TG APM 
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APPENDIX B:\P INFORMATION GUIDE NIGHT VISUAL SCENE EVALUATION 



The objective of this study is to evaluate the training value of a night visual simulator scene for 
daytime aircraft training. As the instructor pilot, you are the primary evaluator. The purpose of this 
guide is to introduce you to the study objectives^ make you aware of the obse^cvations that must be 
made by you to effectively evaluate the student and to help you understand the use of the in^oekpit 
grade cards. 

We are asking that you make rather detailed observations on the student^s performance on three 
maneuvers; takeoff^ straight-in approach and landing, and the straight*in^s in combination with the 
touch-and-go. These observations will be collected by you on the seco nd (B210l)andrifth (B2202) 
sorties. It is essential that the student not have any ^^hands on^ control at these tasks until the second 
{B2101) sortie. In addition to collecting the maneuver specific data, you will be interviewed 
following the snident^s fifth sortie on your subjective comments concerning the student^s 
performance. These interviews are for research purposes only and will be kept strictly confidential. 

Umils of the Ikining in ASPT; 

1. No X*wind training 

2. Little procedural training 

3. Takeoff continued to approximately 1900^ MSL 

4. All landings trained to the center of the runway 

5. Straight*in training initiated at approximately five miles out. 

NOTE : Students who have been trained in ASPT will have had extensive straight-in^ takeoff, and 
tt)uch-and^o trainings so please allow those particular students to fly with as little assistance as 
possible, but of course not '$o com prom i$e flying sofeiyl 

1 have incluficd an enlargement of the takeoff, straighten approach and landings and touch-and* 
go grade cards that you will use in the aircraft^ We would like the data recorded on these cards as soon 
as possible after the student performs the maneuver; however, do not sacrifice clearing or aircraft 
control of any kind to com plctc them. The cards are generally self-explanatory^ but some questions 
have come up regarding unusual circumstances. For example^ during the T/O groundtrack control^ 
you should mark left, on^ or right. If the student docs not over or under control, don^t mark the 
corresponding blocks. On the oihcr hand, if the student docs over or under control^ mark the 
appropriate Uok anil indicate whether loft* on, or right hcst represents the performance. During the 
landing from the straight*in, make sure you indicate whether the flare was smooth or abrupt in 
addition to whether the student flared high, ou, or late. Also* hcsure to indicate whether the straight*^ 
in was a toueh-and*go or full stop. Use your best judgement when filling out the eardsand make sure 
that each item has been marked. Any item not gradci) will make it very difficult for us to evaluate 
your student's performance. If you have to take aircraft control during any part of the maneuver, 
please include it in your comincrits at the bottom of the card. Questions have come up about the 
overall score. Pm stirc you all understand the difference* hut to reinforce your understanding* we 
want the absolute grade to reflect the studciit^s performance against the perfect maneuver. R dative 
score should reflect the performance as a measure of what the student has learned up to this point in 
training. If there are any quostionseoneeming grading or u$c of the eard, please eontaet Dan Cataneo 
at 6601 or Lee Lesher/I)OR 246B. Beft)re you fly with your student on a data ride, please read this 
guide and the (lata cards. If you are nnsiin?, ask! Thank you for your help. 
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APPENDIX C: SUMMARY TASK ITEM ANALYSIS 



I. Takeoff 
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2. Technique 
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ni. Touch^nd^^o (Takeoff portion) 
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